PICO as a Knowledge Representation for Clinical Questions

The paradigm of evidence-based medicine (EBM) recommends that physicians formulate clinical questions in terms of the problem/population, intervention, comparison, and outcome. Together, these elements comprise a PICO frame. Although the framework was developed to facilitate formulation of clinical questions, the ability of PICO structures to represent physicians’ information needs has not been empirically investigated. This paper evaluates the adequacy and suitability of PICO frames as a knowledge representation by analyzing 59 real-life questions in primary care. We discovered that only two questions in our corpus contained all four PICO elements, and that 37% of questions contained only intervention and outcome. Results revealed structural frame patterns that cluster according to the type of clinical question, i.e., therapy, diagnosis, prognosis, and etiology. We found that the PICO framework is primarily centered on therapy questions, and is less suited to representing other types of clinical information needs. Challenges in mapping natural language questions into PICO structures are also discussed. Although we point out limitations of the PICO framework, our study as a whole reaffirms its value as a tool to assist physicians practicing EBM. Introduction Clinicians have from 0.7 to 18.5 questions for every 10 patients cared for. However, answers to twothirds of the questions are either not pursued or pursued but not found. A subsequent analysis shows that almost all unanswered questions could be answered after improved query formulation and search. Therefore, helping doctors to articulate their clinical information needs through well-built, focused questions has become one of the focal points in evidencebased medicine (EBM). EBM provides an explicit framework and guidance for formulating a patientspecific clinical question. According to EBM, articulating a clinical question in terms of its four anatomic parts—Problem/Population, Intervention, Comparison, and Outcome (PICO)—facilitates searching for a precise answer. This study investigates the suitability of the PICO frame as a knowledge representation for clinical questions posed in natural language by practicing physicians. To our knowledge, no research has studied the adequacy and flexibility of the PICO representation and whether it is complete in terms of being able to capture salient characteristics of clinical questions. The present study attempts to address these issues by manually mapping real clinicians’ questions into PICO frames and examining the results. Background A previously-explored approach to understanding the nature of clinical information needs is to collect and to classify real clinical questions from physicians. Through such analyses, studies have introduced question taxonomies in varying levels of details. Taxonomies capable of “covering” a large fraction of clinical questions with a smaller set of “question templates” facilitate access to relevant evidence in the medical literature. Nevertheless, previous studies focus on the surface form of the questions and do not take into account EBM principles for formulating “answerable” clinical questions. The well-built clinical question, focused and wellarticulated in all four components of its anatomy, is widely believed to be the key to efficiently finding the best evidence and also the key to evidence-based decisions. Empirical studies have shown that the use of PICO frames improves the specificity and conceptual breakdown of clinical problems, elicits more information during pre-search reference interview, and leads to more complex search strategies and more precise search results. There are few studies that describe the usability and acceptability of PICO in general, and even less prior work on PICO application in computerized information retrieval systems. A small questionnaire-based study reported a PICO interface for handhelds considered as easy to use and useful in searching MEDLINE. However, the use of PICO-structured frames does not always translate into higher satisfaction. To better understand the adequacy and flexibility of the PICO framework as a knowledge representation, we coded a set of real-world questions asked by physicians into PICO frames. Through the mapping and subsequent analysis, we addressed the following research questions: 1. How well are real-life clinical questions structured according to PICO standards? 2. How well-suited is the PICO frame as a knowledge representation for clinical questions? 3. What concepts and relationships are not adequately captured by PICO representations? 4. Is PICO frame equally suitable for representing different types of clinical questions? Methodology Data Collection We gathered 59 real-world clinical questions from two on-line sources: Family Practice Inquiries Network (FPIN) and Parkhurst Exchange. The question collection process was guided by typical instance sampling rather than random sampling, because the goal is not to obtain a fully representative, but a typical sample of real-life clinical questions. According to the literature, approximately 33% of questions asked by clinicians are about treatment, 25% about diagnosis, and 15% about pharmacotheapeautics. Together, they account for over 70% of clinicians’ questions. Guided by this distribution, four types of clinical questions were gathered: therapy (25), diagnosis (15), prognosis (7), and etiology (12). Coding Clinical Questions with PICO The questions were coded into PICO frames independently by the first and the third author (with backgrounds in library science and medicine, respectively). The comparison and reconciliation of the resulting PICO representations was guided by the second author. This being an exploratory study and the first of its type that we are aware of, the primary purpose of independent coding was to preserve multiple perspectives, rather than to enforce uniformity for the sake of measuring inter-coder agreement. Therefore, no formal instructions or protocol beyond standard EBM guidelines was given to the coders. Analysis of the Results Our corpus of 59 questions was first evaluated for structural completeness. Based on the finding that clinical questions were less likely to go unanswered when the question identified the proposed intervention and desired outcome, we used the cooccurrence of intervention and outcome as an indication of the structural completeness of a question. We then analyzed the prevalence of each PICO element. This analysis gave rise to structural frame patterns that represented prototypical therapy, diagnosis, etiology, and prognosis questions. In addition, semantic classes of concepts present in the 59 clinical questions were identified. This allowed us to construct the typical mapping relationships between semantic entities and PICO elements. * http://www.fpin.org/ † http://www.parkhurstexchange.com/ Finally, challenges encountered during the process of coding these clinical questions were gathered and categorized into emergent themes. This yields a qualitative evaluation of the adequacy of PICO as a knowledge representation for clinical questions. Results Structural Completeness of Clinical Questions In our collected corpus, only two out of 59 questions specified all four PICO elements and 37.3% of questions contain only intervention and outcome. Table 1 provides an overview of how often different PICO elements are presented in each question type. Independent of question type, problem/population and intervention are the most frequently addressed PICO elements (50 and 49 out of 59 respectively), followed by population (29 out of 59), then by outcome (27 out of 59). In contrast, comparison is rarely mentioned (only 3 out of 59). Table 1. Structural Completeness for Four Types of Clinical Questions. 1–number of questions; 2– all elements present; 3–intervention and outcome present Therapy Diagnosis Etiology Prognosis Total